BORG: Block-reORGanization and Self-optimization in Storage Systems

Disk I/O is the primary performance bottleneck for a wide range of workloads due to the relatively large, mechanical seek and rotational delay overheads incurred during I/O operations. Current-day file systems that exclusively manage storage space on disk drives employ static data layouts and do not attempt to optimize for application access patterns. We argue that this lack of application awareness in operating system storage management is one of the key reasons for sub-optimal disk I/O performance. We present the design and implementation of BORG, a self-optimizing block storage layer that performs automatic block reorganization to optimize storage system performance, while remaining oblivious to the file system(s) and application layers above and the I/O scheduling and device driver layers below. BORG optimizes storage system performance in a continuous and online fashion by dynamically reorganizing disk data to best suit common disk access patterns. It successfully addresses the key requirements for a self-optimizing storage solution, incorporating accurate extraction and representation of disk access patterns, file system independence, modularization of storage stack layers, isolation of space management responsibilities, data consistency, online optimization capability, and overhead control. A Linux implementation of BORG demonstrates consistent improvements in disk I/O performance for a variety of workloads ranging from file servers and web servers to development workstations and even individual desktop applications, with acceptable overhead in other resource dimensions.